Exact Algorithms for Solving Stochastic Games

Shapley's discounted stochastic games, Everett's recursive games and Gillette's undiscounted stochastic games are classical models of game theory describing two-player zero-sum games of potentially infinite duration. We describe algorithms for exactly solving these games

Recursive Stochastic Effects in Valley Hybrid Inflation

Hybrid Inflation is a two-field model where inflation ends by a tachyonic instability, the duration of which is determined by stochastic effects and has important observational implications. Making use of the recursive approach to the stochastic formalism presented in Ref. [1], these effects are consistently computed. Through an analysis of back-reaction, this method is shown to converge in the valley but points toward an (expected) instability in the waterfall. It is further shown that quasi-stationarity of the auxiliary field distribution breaks down in the case of a short-lived waterfall. It is found that the typical dispersion of the waterfall field at the critical point is then diminished, thus increasing the duration of the waterfall phase and jeopardizing the possibility of a short transition. Finally, it is found that stochastic effects worsen the blue tilt of the curvature perturbations by an order one factor when compared with the usual slow-roll contribution.Comment: 26 pages, 6 figure

Bayesian Analysis of the Stochastic Conditional Duration Model

A Bayesian Markov Chain Monte Carlo methodology is developed for estimating the stochastic conditional duration model. The conditional mean of durations between trades is modelled as a latent stochastic process, with the conditional distribution of durations having positive support. The sampling scheme employed is a hybrid of the Gibbs and Metropolis Hastings algorithms, with the latent vector sampled in blocks. The suggested approach is shown to be preferable to the quasi-maximum likelihood approach, and its mixing speed faster than that of an alternative single-move algorithm. The methodology is illustrated with an application to Australian intraday stock market data.Transaction data, Latent factor model, Non-Gaussian state space model, Kalman filter and simulation smoother.

Operator approach to values of stochastic games with varying stage duration

We study the links between the values of stochastic games with varying stage duration $h$, the corresponding Shapley operators $\bf{T}$ and ${\bf{T}}\_h$and the solution of $\dot f\_t = ({\bf{T}} - Id )f\_t$. Considering general non expansive maps we establish two kinds of results, under both the discounted or the finite length framework, that apply to the class of "exact" stochastic games. First, for a fixed length or discount factor, the value converges as the stage duration go to 0. Second, the asymptotic behavior of the value as the length goes to infinity, or as the discount factor goes to 0, does not depend on the stage duration. In addition, these properties imply the existence of the value of the finite length or discounted continuous time game (associated to a continuous time jointly controlled Markov process), as the limit of the value of any time discretization with vanishing mesh.Comment: 22 pages, International Journal of Game Theory, Springer Verlag, 201

Nonlinear Stochastic Resonance with subthreshold rectangular pulses

We analyze the phenomenon of nonlinear stochastic resonance (SR) in noisy bistable systems driven by pulsed time periodic forces. The driving force contains, within each period, two pulses of equal constant amplitude and duration but opposite signs. Each pulse starts every half-period and its duration is varied. For subthreshold amplitudes, we study the dependence of the output signal-to-noise ratio (SNR) and the SR gain on the noise strength and the relative duration of the pulses. We find that the SR gains can reach values larger than unity, with maximum values showing a nonmonotonic dependence on the duration of the pulses.Comment: 7 pages, 2 figure

Robust Stochastic Chemical Reaction Networks and Bounded Tau-Leaping

The behavior of some stochastic chemical reaction networks is largely unaffected by slight inaccuracies in reaction rates. We formalize the robustness of state probabilities to reaction rate deviations, and describe a formal connection between robustness and efficiency of simulation. Without robustness guarantees, stochastic simulation seems to require computational time proportional to the total number of reaction events. Even if the concentration (molecular count per volume) stays bounded, the number of reaction events can be linear in the duration of simulated time and total molecular count. We show that the behavior of robust systems can be predicted such that the computational work scales linearly with the duration of simulated time and concentration, and only polylogarithmically in the total molecular count. Thus our asymptotic analysis captures the dramatic speedup when molecular counts are large, and shows that for bounded concentrations the computation time is essentially invariant with molecular count. Finally, by noticing that even robust stochastic chemical reaction networks are capable of embedding complex computational problems, we argue that the linear dependence on simulated time and concentration is likely optimal

Spurious memory in non-equilibrium stochastic models of imitative behavior

The origin of the long-range memory in the non-equilibrium systems is still an open problem as the phenomenon can be reproduced using models based on Markov processes. In these cases a notion of spurious memory is introduced. A good example of Markov processes with spurious memory is stochastic process driven by a non-linear stochastic differential equation (SDE). This example is at odds with models built using fractional Brownian motion (fBm). We analyze differences between these two cases seeking to establish possible empirical tests of the origin of the observed long-range memory. We investigate probability density functions (PDFs) of burst and inter-burst duration in numerically obtained time series and compare with the results of fBm. Our analysis confirms that the characteristic feature of the processes described by a one-dimensional SDE is the power-law exponent $3/2$ of the burst or inter-burst duration PDF. This property of stochastic processes might be used to detect spurious memory in various non-equilibrium systems, where observed macroscopic behavior can be derived from the imitative interactions of agents.Comment: 11 pages, 5 figure

Statistical analysis of the equivalent design rainfall

Statistical analyses of rainfall data are used for the design of sewerage systems and pump-stations, for the evaluation of the duration and the frequency of overflow in runoff detention facilities, for the determination of the critical influence on a municipal wastewater-treatment plant or for the protection of watercourses from storm-water runoff (e.g., from highways). The basic data in this calculation are the intensity and the duration of a rainstorm. Different procedures used in the analysis of Equivalent Design Rainfall (EDR) in Slovenia and abroad are described. The stochastic model used is presented in more detail because of its applicability for the determination of the probability of the occurrence of partial rainfalls of higher frequencies and the determination of the lower limit of rainfall evaluation. Computation procedures and the results of the evaluation of rainfall data according to the stochastic model are presented for Ljubljana